Papers with document classification
HarriGT: A Tool for Linking News to Science (P18-4)
Copied to clipboard
| Challenge: | citation-based metrics are not as transparent as once thought, says a new study . citation metrics are a key component of measuring scientific impact in society . |
| Approach: | They propose a tool to build corpora of news articles linked to scientific papers . citation-based metrics have catalysed research funding councils' interest in impact . |
| Outcome: | The proposed tool can build corpora of news articles linked to scientific papers . it integrates with 3 large external citation networks to surface relevant examples of scientific literature . |
Joint Training for Learning Cross-lingual Embeddings with Sub-word Information without Parallel Corpora (2020.starsem-1)
Copied to clipboard
| Challenge: | Existing methods for learning cross-lingual word embeddings incorporate sub-word information during training. |
| Approach: | They propose a method that incorporates sub-word information during training to learn cross-lingual word embeddings from monolingual data and a bilingual lexicon. |
| Outcome: | The proposed method improves on bilingual lexicon induction, monolingual word similarity, and document classification using low-resource languages. |
Neural Speed Reading Audited (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Several approaches to neural speed reading have been presented at major NLP and machine learning conferences in 2017–20. |
| Approach: | They propose to model "human speed reading" for more efficient NLP, including document classification and named entity recognition. |
| Outcome: | The proposed approach has 7% error reduction and 136x speed-up over the state-of-the-art in neural speed reading. |
Can Large Language Models Serve as Effective Classifiers for Hierarchical Multi-Label Classification of Scientific Documents at Industrial Scale? (2025.coling-industry)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated great potential in complex tasks such as multi-label classification, but the vast number of labels can exceed LLMs’ input limits. |
| Approach: | They propose a method that integrates large language models with dense retrieval techniques to overcome these challenges. |
| Outcome: | The proposed methods avoid frequent retraining by leveraging zero-shot and few-shot learning for real-time label assignment. |
Towards Comprehensive Patent Approval Predictions:Beyond Traditional Document Classification (2022.acl-long)
Copied to clipboard
| Challenge: | a new framework for patent approval prediction is proposed to address this problem . novelty scores are based on comparing an application with millions of prior arts . |
| Approach: | They propose a framework that unifies the document classifier with handcrafted features, particularly time-dependent novelty scores. |
| Outcome: | The proposed framework unifies the document classifier with handcrafted features, particularly time-dependent novelty scores. |
GAINER: Graph Machine Learning with Node-specific Radius for Classification of Short Texts and Documents (2024.eacl-long)
Copied to clipboard
| Challenge: | Recent advances in Graph Machine Learning (GML) have led to the development of numerous models tailored for processing text for various natural language applications. |
| Approach: | They propose a framework called Graph mAchine learnIng with Node-spEcific Radius that is aimed at graph-based NLP. |
| Outcome: | The proposed framework is non-neural and novel for graph-based NLP. |
Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated Pooling Mechanisms (P18-1)
Copied to clipboard
Dinghan Shen, Guoyin Wang, Wenlin Wang, Martin Renqiang Min, Qinliang Su, Yizhe Zhang, Chunyuan Li, Ricardo Henao, Lawrence Carin
| Challenge: | Existing deep learning architectures to model compositionality in text sequences require a large number of parameters and expensive computations. |
| Approach: | They propose two additional pooling strategies over word embeddings for improved interpretability and hierarchical pooling for spatial (n-gram) information within text sequences. |
| Outcome: | The proposed pooling strategies improve interpretability and preserve spatial (n-gram) information within text sequences. |
Unsupervised Learning of Discourse-Aware Text Representation for Essay Scoring (P19-2)
Copied to clipboard
| Challenge: | Existing document embedding approaches focus on capturing sequences of words in documents . however, some document classification and regression tasks need to consider discourse structure of text . |
| Approach: | They propose an unsupervised approach to capture discourse structure in terms of coherence and cohesion for document embedding that does not require expensive parsers or annotation. |
| Outcome: | The proposed method improves essay Organization scoring and Argument Strength scoring. |
Cascaded Semantic and Positional Self-Attention Network for Document Classification (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to document classification combine semantic information with positional information (word orders) . document classification is one of the fundamental problems in natural language processing . |
| Approach: | They propose a new architecture to combine semantic and positional information using a semantic self-attention layer cascaded with Bi-LSTM. |
| Outcome: | The proposed model can exploit the interaction between semantics and word positions in a more interpretable and adaptive manner while preserving a compact model size and high convergence rate. |
Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT (D19-1)
Copied to clipboard
| Challenge: | Pretrained contextual representation models have pushed forward the state-of-the-art on many NLP tasks. |
| Approach: | They propose to use a model that is pretrained on 104 languages for cross-lingual transfer. |
| Outcome: | The proposed model performs well on 5 NLP tasks covering 39 languages from various language families. |
K-hop neighbourhood regularization for few-shot learning on graphs: A case study of text classification (2023.eacl-main)
Copied to clipboard
| Challenge: | We show that few-sample word-document graphs can be used for improved learning in low-resource settings. |
| Approach: | They propose a method to utilize word-document graph properties for improved learning in low-resource settings by using a regularizer for heterogeneous graphs. |
| Outcome: | The proposed method outperforms a baseline TextGCN with 17% accuracy over eight languages while performing on par with the state-of-the-art models. |
Integration of Knowledge Graph Embedding Into Topic Modeling with Hierarchical Dirichlet Process (N19-1)
Copied to clipboard
| Challenge: | Topic models are used to extract topical structures from document-word frequency representations of the text corpus without supervision. |
| Approach: | They propose a Bayesian nonparametric topic modeling with knowledge graph embedding to employ knowledge graphs to extract more coherent topics. |
| Outcome: | The proposed model performs better on three public datasets than state-of-the-art models on topic coherence and document classification accuracy. |
Tsetlin Machine Embedding: Representing Words Using Logical Expressions (2024.findings-eacl)
Copied to clipboard
| Challenge: | Embedding words in vector space is a fundamental first step in state-of-the-art natural language processing. |
| Approach: | They propose to embed words in vector space using propositional logic instead of dense vectors . they evaluate embeddings on intrinsic and extrinsic benchmarks and visualize word clusters based on their results . |
| Outcome: | The proposed model outperforms GLoVe on six classification tasks. |
SubRegWeigh: Effective and Efficient Annotation Weighing with Subword Regularization (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods to reduce the adverse effect of annotation errors are time-consuming because they require many trained models to detect errors. |
| Approach: | They propose a method that uses a tokenization technique called subword regularization to simulate multiple error detection models for detecting errors. |
| Outcome: | The proposed method performs weighting weighting four to five times faster than existing methods and improves in document classification and named entity recognition tasks. |
Are Your Keywords Like My Queries? A Corpus-Wide Evaluation of Keyword Extractors with Real Searches (2025.coling-main)
Copied to clipboard
| Challenge: | Keyword Extraction (KE) is essential in Natural Language Processing (NLP) for identifying key terms that represent the main themes of a text. |
| Approach: | They propose to use real query data from Google Trends to evaluate keywords extracted from a text to capture users' top queries. |
| Outcome: | The proposed method can be used with both supervised and unsupervised KE approaches and shows that KeyBERT is the most effective in capturing users’ top queries. |
TopicBERT for Energy Efficient Document Classification (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work has noted that BERT’s computational cost grows quadratically with sequence length thus leading to longer training times, higher GPU memory constraints and carbon emissions. |
| Approach: | They propose to combine topic and language models to optimize the computational cost of fine-tuning for document classification by complementary learning. |
| Outcome: | The proposed model achieves a 1.4x speedup with 40% reduction in CO2 emission while retaining 99.9% performance over 5 datasets. |
Word Tour: One-dimensional Word Embeddings via the Traveling Salesman Problem (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing word embeddings are high-dimensional and consume considerable computational resources. |
| Approach: | They propose a method to decompose the desiderata of word embeddings into two parts, completeness and soundness, and focus on soundness. |
| Outcome: | The proposed method is extremely efficient and provides minimal means to handle word embeddings. |
Multilingual and cross-lingual document classification: A meta-learning approach (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing methods to document classification in low-resource languages are under-resourced . 6% of the world's languages are spoken, and many have inadequate resources . |
| Approach: | They propose a meta-learning approach to document classification in low-resource languages . they propose 'nuclear-shot' cross-lingual adaptation to previously unseen languages based on limited data . |
| Outcome: | The proposed method performs on-par on some languages while under-resourced in others. |
Improved Multi-label Classification under Temporal Concept Drift: Rethinking Group-Robust Algorithms in a Label-Wise Setting (2022.findings-acl)
Copied to clipboard
| Challenge: | Large-scale multi-label document classification presents interesting challenges due to the large label space and two-tiered skewed label distributions. |
| Approach: | They evaluate several group-robust optimization algorithms proposed to mitigate temporal concept drift and class imbalance in document classification. |
| Outcome: | The proposed algorithms outperform sampling-based approaches to class imbalance and concept drift and lead to much better performance on minority classes. |
On Evaluation of Document Classification with RVL-CDIP (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing document classification benchmarks have label noise, ambiguous documents, and sensitive information. |
| Approach: | They argue that RVL-CDIP is unsuitable for benchmarking document classifiers . they advocate for a new document classification benchmark with ambiguous labels . |
| Outcome: | The RVL-CDIP benchmark is widely used for document classification . the authors argue that its limited scope, presence of errors and lack of diversity make it less than ideal for benchmarking. |
SPECTER: Document-level Representation Learning using Citation-informed Transformers (2020.acl-main)
Copied to clipboard
| Challenge: | Recent Transformer language models do not leverage information on inter-document relatedness, which limits their document-level representation power. |
| Approach: | They propose a method to generate document-level embeddings using citation graphs. |
| Outcome: | The proposed method outperforms baselines on document-level tasks. |
Improving Conversational Question Answering Systems after Deployment using Feedback-Weighted Learning (2020.coling-main)
Copied to clipboard
| Challenge: | In most applications, users are not able to provide the correct answer to the system, but they are able provide binary (correct, incorrect) feedback. |
| Approach: | They propose feedback-weighted learning based on importance sampling to improve upon an initial supervised system using binary user feedback. |
| Outcome: | The proposed method improves on an initial supervised system, getting close to a fully-supervised system that has access to the same labeled examples in in-domain experiments (QuAC) and matching in out-of-domain experiment (DoQA). |
Neural Generative Rhetorical Structure Parsing (D19-1)
Copied to clipboard
| Challenge: | Existing approaches to RST parsing use discriminative models, but these are less sample efficient and datasets are typically small. |
| Approach: | They propose a document-level RNN grammar with a bottom-up traversal order and a beam search algorithm that keeps track of both structure-and word-generating actions without exhibit-ing this branching bias. |
| Outcome: | The proposed model outperforms a discriminative model with the same features by 2.6 F1points and achieves comparable performance to the state-of-the-art. |
Low-resource Interactive Active Labeling for Fine-tuning Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing active learning methods for fine-tuning language models are underperforming in low-resource, interactive labeling setting. |
| Approach: | They propose a novel active learning method that employs a hybrid sampling strategy to minimize labeling cost and acquisition latency while providing a framework for adapting to dataset diversity. |
| Outcome: | The proposed method reduces labeling cost and acquisition latency while providing a framework for adapting to dataset diversity via user guidance. |
Exploring the Relationship Between Algorithm Performance, Vocabulary, and Run-Time in Text Classification (2021.naacl-main)
Copied to clipboard
| Challenge: | Many text classification algorithms depend on the size of the corpus’ vocabulary due to their bag-of-words representation. |
| Approach: | They propose to evaluate how preprocessing techniques affect the run-time of models by evaluating ten techniques over four models and two datasets. |
| Outcome: | The proposed methods can reduce run-time with no loss of accuracy while sacrificing up to 65%. |
Using J-K-fold Cross Validation To Reduce Variance When Tuning NLP Models (C18-1)
Copied to clipboard
| Challenge: | a recent study shows that performance estimations are unstable and variable . this makes it difficult to use parameter tuning and model selection . |
| Approach: | They propose to use a less variable CV method to evaluate performance . they propose lower choices of K than are typically seen in the NLP literature . |
| Outcome: | The proposed method can be used for parameter tuning and performance estimation, but it is unstable and unstable. |
Self-Discriminative Learning for Unsupervised Document Embedding (N19-1)
Copied to clipboard
| Challenge: | Existing methods for document embedding learning do not consider inter-document relationships. |
| Approach: | They propose to exploit the inter-document information and directly model the relations of documents in embedding space with a discriminative network and a novel objective. |
| Outcome: | The proposed method has errors that are 5 to 13% lower than state-of-the-art models and is even more pronounced in scarce label setting. |
Visually Guided Generative Text-Layout Pre-training for Document Intelligence (2024.naacl-long)
Copied to clipboard
| Challenge: | Prior work shows that pre-training techniques can boost the performance of visual document understanding (VDU) . Xu et al., 2020;; Gu e t al, 2021;; Appalaraju e al. 2022) |
| Approach: | They propose a visually guided generative text-layout pre-training method that optimizes hierarchical language and layout modeling objectives to generate interleaved text and layout sequences. |
| Outcome: | The proposed model can process word-intensive documents of any length and achieves competitive performance over baselines on VDU tasks. |
TAN-NTM: Topic Attention Networks for Neural Topic Modeling (2021.acl-long)
Copied to clipboard
| Challenge: | Topic models have been widely used to learn text representations and gain insight into document corpora. |
| Approach: | They propose a framework which processes document as a sequence of tokens through a LSTM whose contextual outputs are attended in a topic-aware manner. |
| Outcome: | The proposed model improves on two downstream tasks: document classification and topic guided keyphrase generation. |
Contrastive Document Representation Learning with Graph Attention Networks (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for document representation learning are significantly affected by the scarcity of document-level data. |
| Approach: | They propose to use a graph attention network on top of the available pretrained Transformers model to learn document embeddings. |
| Outcome: | Empirically, the proposed approach is effective in document classification and document retrieval tasks. |
ContraDoc: Understanding Self-Contradictions in Documents with Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Detecting contradictions in texts is often regarded as determining relation between hypothesis and piece of premise. |
| Approach: | They propose a human-annotated dataset to study self-contradictions in long documents . they analyze the capabilities of four open-source and commercially available LLMs . |
| Outcome: | The proposed dataset outperforms open-source LLMs on document-level tasks but struggles with self-contradictions that require more nuance and context. |
Disentangling Structure and Style: Political Bias Detection in News by Inducing Document Hierarchy (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a new method to detect political bias in news articles overcomes this domain dependency . partisan bias exists in various social issues, including the 2016 presidential election . |
| Approach: | They propose a multi-head hierarchical attention model that encodes the structure of long documents through a diverse ensemble of attention heads. |
| Outcome: | The proposed model outperforms existing methods for detecting political bias in news articles. |
Neural Temporality Adaptation for Document Classification: Diachronic Word Embeddings and Domain Adaptation Models (P19-1)
Copied to clipboard
| Challenge: | Recent studies show that document classifiers can become more stable over time when trained in ways that account for temporal variations. |
| Approach: | They propose a method for embedding diachronic word embedds into document classification models . they propose 'time-driven neural classification model' that accounts for temporal variations . |
| Outcome: | The proposed model can be trained on six corpora and make it more robust over time. |
Rethinking Complex Neural Network Architectures for Document Classification (N19-1)
Copied to clipboard
| Challenge: | Neural network models for many NLP tasks have grown increasingly complex in recent years . authors of recent papers question the necessity of such architectures and find them quite effective . |
| Approach: | They propose to use regularization techniques borrowed from language modeling to improve model accuracy . they find that a simple biLSTM architecture with appropriate regularization yields competitive results . |
| Outcome: | a simple biLSTM model outperforms the state-of-the-art on four benchmark datasets . authors say that improvements are not real, but are attributed to mundane reasons . |
MDACE: MIMIC Documents Annotated with Code Evidence (2023.acl-long)
Copied to clipboard
Hua Cheng, Rana Jafari, April Russell, Russell Klopfer, Edmond Lu, Benjamin Striner, Matthew Gormley
| Challenge: | Computer-Assisted Coding (CAC) systems are required to provide supporting textual evidence to justify billing codes. |
| Approach: | They propose a dataset for evidence/rationale extraction on an extreme multi-label classification task over long medical documents. |
| Outcome: | The proposed dataset can be used to evaluate evidence extraction methods for CAC systems, as well as the accuracy and interpretability of deep learning models for multi-label classification. |
SuperGLEBer: German Language Understanding Evaluation Benchmark (2024.naacl-long)
Copied to clipboard
| Challenge: | a new set of German-pretrained models are being released, but no established, diverse and systematic evaluation suite is available for them. |
| Approach: | They assemble a Natural Language Understanding benchmark suite for the German language and evaluate 10 existing German-pretrained models. |
| Outcome: | The proposed benchmark suite evaluates 10 German-pretrained models on 29 tasks . the results show that encoder models are good choices for most tasks, but not all . |
Privacy at Scale: Introducing the PrivaSeer Corpus of Web Privacy Policies (2021.acl-long)
Copied to clipboard
| Challenge: | Existing tools to interpret privacy policies have been used to understand them but there is a lack of large privacy policy corpora to simplify the process. |
| Approach: | They propose to use a corpus of 1,005,380 English language privacy policies collected from the web to create semi-supervised and unsupervised models to interpret and simplify privacy policies. |
| Outcome: | The proposed model outperforms all other publicly available privacy policy corpora and is ten times larger than the next largest public collection of privacy policies combined. |
Revisiting Transformer-based Models for Long Document Classification (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent literature in text classification is biased towards short text sequences . multi-page multi-paragraph documents cannot be efficiently encoded by vanilla transformers based on short text. |
| Approach: | They compare different Transformer-based Long Document Classification approaches to mitigate the computational overhead of vanilla transformers to encode much longer text. |
| Outcome: | The proposed models can process longer text and provide practical advice for long document classification tasks. |
German’s Next Language Model (2020.coling-main)
Copied to clipboard
| Challenge: | In this paper we compare the performance of our deep transformer based language models to existing models. |
| Approach: | They present a set of BERT and ELECTRA based German language models, GBERT and GELECTRE. |
| Outcome: | The proposed models outperform the previous best models on NER and classification tasks but are prohibitively large for many. |
Text Classification Using Label Names Only: A Language Model Self-Training Approach (2020.emnlp-main)
Copied to clipboard
| Challenge: | Current text classification methods require a large number of labeled documents as training data. |
| Approach: | They propose a model that uses only the label name of each class to train classification models on unlabeled data without using any labeled examples. |
| Outcome: | The proposed model achieves 90% accuracy on four benchmark datasets using label names as the only supervision . |
Large-scale similarity search with Optimal Transport (2023.emnlp-main)
Copied to clipboard
| Challenge: | Word mover's distance (WMD) is a powerful tool for comparing probability distributions in NLP. |
| Approach: | They propose a waterstein distance approximation that uses the L1 embedding method to find the k-nearest neighbors. |
| Outcome: | The proposed approximation performs comparable to the vanilla Wasserstein distance and can be computed three orders of magnitude faster than the vanilla waterstein distance. |
Human in the Loop: How to Effectively Create Coherent Topics by Manually Labeling Only a Few Documents per Class (2024.lrec-main)
Copied to clipboard
| Challenge: | Few-shot methods for accurate modeling under sparse label-settings are still challenging in document classification. |
| Approach: | They propose to combine supervised few-shot learning with a topic extraction method to generate coherent topics in large text corpora. |
| Outcome: | The proposed method outperforms unsupervised topic modeling methods in document classification. |
Softmax Tree: An Accurate, Fast Classifier When the Number of Classes Is Large (2021.emnlp-main)
Copied to clipboard
| Challenge: | Classification problems with thousands or more classes occur in NLP, for example language models or document classification. |
| Approach: | a new algorithm uses a binary tree with sparse hyperplanes and small softmax classifiers at the leaves to predict the top class. |
| Outcome: | The proposed model is faster at inference because the input follows a single path to a leaf and the softmax classifier operates on a small subset of the classes. |
A linear time approximation of Wasserstein distance with word embedding selection (2023.emnlp-main)
Copied to clipboard
| Challenge: | Wasserstein distance is a powerful method for measuring the dissimilarity between distributions and is used in natural language processing to measure dissimilarities between documents. |
| Approach: | They propose a method to combine feature selection and tree approximation of Wasserstein distance to handle high-dimensional problems. |
| Outcome: | The proposed method achieves high performance on document classification using word embeddings and word embeds. |
DocSplit: Simple Contrastive Pretraining for Large Document Embeddings (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing model pretraining methods only consider local information, resulting in low-quality embeddings for large documents. |
| Approach: | They propose a new method which forces models to consider the entire global context of a large document. |
| Outcome: | The proposed method outperforms existing models on document classification, few shot learning, and retrieval tasks. |
IRIS: Interpretable Retrieval-Augmented Classification for Long Interspersed Document Sequences (2025.acl-long)
Copied to clipboard
| Challenge: | Existing models for document classification struggle with long-text processing due to quadratic computational complexity in the self-attention module. |
| Approach: | They propose a framework that utilizes retrieval to efficiently classify long documents . they use a quadratic attention matrix to capture dependencies between tokens in an input sequence . |
| Outcome: | The proposed framework excels in clinical note disease risk prediction tasks . it can process arbitrarily long documents without increasing computational cost and trainable on a single GPU. |